单调可行的算法的开发,受基数约束(SMCC)的基本最大化产生了两个单独的研究方向:具有低自适应复杂性的集中算法,需要随机访问整个数据集;并分布式MAPREDUCE(MR)模型算法,这些算法使用少量的MR回合计算。目前,众所周知,没有MR Model算法使用均值的自适应回合,从而限制了其实际性能。我们在分布式设置中研究了SMCC问题,并介绍了三种单独的MR模型算法,这些算法在分布式设置中引入了sublinear适应性。我们的主要算法,Dash实现了$ \ frac {1} {2} {2}(1-1/e- \ varepsilon)$的近似值,而使用一个MR圆形,而其多轮变体元数据启用MR模型算法可以在大型上运行。以前不可能的基数约束。使用一个和$($ \ frac {3} {8} {8} - \ varepsilon $)和($ 1-1/e- \ varepsilon $)的两种附加算法T-DASH和G-DASH提供了改进的比率为($ \ frac {3} {8} - \ varepsilon $) 1/\ Varepsilon)$ MR ROUNDS。我们所有提出的算法都具有肌关系的自适应复杂性,我们提供了广泛的经验证据来确定:仪表率是比最先进的分布式算法快的数量级,同时产生了几乎相同的溶液值;并验证仪表板在集中和分布式数据上获得可行解决方案时的多功能性。
translated by 谷歌翻译
对于最大化单调的问题,子模块功能相对于基数限制为$ K $ k $ k $ k $ $ n $ n $,我们提供了一种在其经验性能和其上实现最先进的算法理论属性,就适应性复杂性,查询复杂性和近似率而言;也就是说,它获得了高概率,查询复杂度$ O(n)$的期望,适应$ o(\ log(n))$,近似1-1 / e $的近似比。主要算法由可能是独立兴趣的两个组件组装。我们的算法的第一个组件LineArseq,可用作提高许多算法的查询复杂性的预处理算法。此外,LineArseq的变体显示为具有O $ O(n / k))$的自适应复杂性,其小于文献中的任何先前算法的自适应复杂性。第二组件是一个并行阈值处理过程阈值问题,用于添加具有高于恒定阈值的增益的元素。最后,我们展示了我们的主要算法在运行时,自适应轮次,总查询和客观值方面经验胜过,以前的最先进的算法,以六个子模块物理函数快速评估。
translated by 谷歌翻译
Transformer layers, which use an alternating pattern of multi-head attention and multi-layer perceptron (MLP) layers, provide an effective tool for a variety of machine learning problems. As the transformer layers use residual connections to avoid the problem of vanishing gradients, they can be viewed as the numerical integration of a differential equation. In this extended abstract, we build upon this connection and propose a modification of the internal architecture of a transformer layer. The proposed model places the multi-head attention sublayer and the MLP sublayer parallel to each other. Our experiments show that this simple modification improves the performance of transformer networks in multiple tasks. Moreover, for the image classification task, we show that using neural ODE solvers with a sophisticated integration scheme further improves performance.
translated by 谷歌翻译
This paper presents SVAM (Sequential Variance-Altered MLE), a unified framework for learning generalized linear models under adversarial label corruption in training data. SVAM extends to tasks such as least squares regression, logistic regression, and gamma regression, whereas many existing works on learning with label corruptions focus only on least squares regression. SVAM is based on a novel variance reduction technique that may be of independent interest and works by iteratively solving weighted MLEs over variance-altered versions of the GLM objective. SVAM offers provable model recovery guarantees superior to the state-of-the-art for robust regression even when a constant fraction of training labels are adversarially corrupted. SVAM also empirically outperforms several existing problem-specific techniques for robust regression and classification. Code for SVAM is available at https://github.com/purushottamkar/svam/
translated by 谷歌翻译
Search and rescue, wildfire monitoring, and flood/hurricane impact assessment are mission-critical services for recent IoT networks. Communication synchronization, dependability, and minimal communication jitter are major simulation and system issues for the time-based physics-based ROS simulator, event-based network-based wireless simulator, and complex dynamics of mobile and heterogeneous IoT devices deployed in actual environments. Simulating a heterogeneous multi-robot system before deployment is difficult due to synchronizing physics (robotics) and network simulators. Due to its master-based architecture, most TCP/IP-based synchronization middlewares use ROS1. A real-time ROS2 architecture with masterless packet discovery synchronizes robotics and wireless network simulations. A velocity-aware Transmission Control Protocol (TCP) technique for ground and aerial robots using Data Distribution Service (DDS) publish-subscribe transport minimizes packet loss, synchronization, transmission, and communication jitters. Gazebo and NS-3 simulate and test. Simulator-agnostic middleware. LOS/NLOS and TCP/UDP protocols tested our ROS2-based synchronization middleware for packet loss probability and average latency. A thorough ablation research replaced NS-3 with EMANE, a real-time wireless network simulator, and masterless ROS2 with master-based ROS1. Finally, we tested network synchronization and jitter using one aerial drone (Duckiedrone) and two ground vehicles (TurtleBot3 Burger) on different terrains in masterless (ROS2) and master-enabled (ROS1) clusters. Our middleware shows that a large-scale IoT infrastructure with a diverse set of stationary and robotic devices can achieve low-latency communications (12% and 11% reduction in simulation and real) while meeting mission-critical application reliability (10% and 15% packet loss reduction) and high-fidelity requirements.
translated by 谷歌翻译
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.
translated by 谷歌翻译
Recent research has demonstrated the capability of behavior signals captured by smartphones and wearables for longitudinal behavior modeling. However, there is a lack of a comprehensive public dataset that serves as an open testbed for fair comparison among algorithms. Moreover, prior studies mainly evaluate algorithms using data from a single population within a short period, without measuring the cross-dataset generalizability of these algorithms. We present the first multi-year passive sensing datasets, containing over 700 user-years and 497 unique users' data collected from mobile and wearable sensors, together with a wide range of well-being metrics. Our datasets can support multiple cross-dataset evaluations of behavior modeling algorithms' generalizability across different users and years. As a starting point, we provide the benchmark results of 18 algorithms on the task of depression detection. Our results indicate that both prior depression detection algorithms and domain generalization techniques show potential but need further research to achieve adequate cross-dataset generalizability. We envision our multi-year datasets can support the ML community in developing generalizable longitudinal behavior modeling algorithms.
translated by 谷歌翻译
In this research work, we have demonstrated the application of Mask-RCNN (Regional Convolutional Neural Network), a deep-learning algorithm for computer vision and specifically object detection, to semiconductor defect inspection domain. Stochastic defect detection and classification during semiconductor manufacturing has grown to be a challenging task as we continuously shrink circuit pattern dimensions (e.g., for pitches less than 32 nm). Defect inspection and analysis by state-of-the-art optical and e-beam inspection tools is generally driven by some rule-based techniques, which in turn often causes to misclassification and thereby necessitating human expert intervention. In this work, we have revisited and extended our previous deep learning-based defect classification and detection method towards improved defect instance segmentation in SEM images with precise extent of defect as well as generating a mask for each defect category/instance. This also enables to extract and calibrate each segmented mask and quantify the pixels that make up each mask, which in turn enables us to count each categorical defect instances as well as to calculate the surface area in terms of pixels. We are aiming at detecting and segmenting different types of inter-class stochastic defect patterns such as bridge, break, and line collapse as well as to differentiate accurately between intra-class multi-categorical defect bridge scenarios (as thin/single/multi-line/horizontal/non-horizontal) for aggressive pitches as well as thin resists (High NA applications). Our proposed approach demonstrates its effectiveness both quantitatively and qualitatively.
translated by 谷歌翻译
Long-range context modeling is crucial to both dialogue understanding and generation. The most popular method for dialogue context representation is to concatenate the last-$k$ previous utterances. However, this method may not be ideal for conversations containing long-range dependencies. In this work, we propose DialoGX, a novel encoder-decoder based framework for conversational response generation with a generalized and explainable context representation that can look beyond the last-$k$ utterances. Hence the method is adaptive to conversations with long-range dependencies. The main idea of our approach is to identify and utilize the most relevant historical utterances instead of the last-$k$ utterances in chronological order. We study the effectiveness of our proposed method on both dialogue generation (open-domain) and understanding (DST) tasks. DialoGX achieves comparable performance with the state-of-the-art models on DailyDialog dataset. We also observe performance gain in existing DST models with our proposed context representation strategy on MultiWOZ dataset. We justify our context representation through the lens of psycholinguistics and show that the relevance score of previous utterances agrees well with human cognition which makes DialoGX explainable as well.
translated by 谷歌翻译
肾脏是人体的重要器官。它保持体内平衡并通过尿液去除有害物质。肾细胞癌(RCC)是肾癌最常见的形式。大约90%的肾脏癌归因于RCC。最有害的RCC类型是清晰的细胞肾细胞癌(CCRCC),占所有RCC病例的80%。需要早期和准确的CCRCC检测,以防止其他器官进一步扩散该疾病。在本文中,进行了详细的实验,以确定可以在不同阶段诊断CCRCC的重要特征。 CCRCC数据集从癌症基因组图集(TCGA)获得。考虑了从8种流行特征选择方法获得的特征顺序的新型相互信息和集合的特征排名方法。通过使用2个不同的分类器(ANN和SVM)获得的总体分类精度来评估所提出方法的性能。实验结果表明,所提出的特征排名方法能够获得更高的精度(分别使用SVM和NN分别使用SVM和NN),与现有工作相比,使用SVM和NN分别使用SVM和NN进行分类。还要注意的是,在现有TNM系统(由AJCC和UICC提出的)提到的3个区分特征中,我们提出的方法能够选择其中两个(肿瘤的大小,转移状态)作为顶部 - 大多数。这确立了我们提出的方法的功效。
translated by 谷歌翻译